Spiking neural network system for dynamic control of flexible, stable and hybrid memory storage

ABSTRACT

Provided is a spiking neural network system for dynamical control of flexible, stable, and hybrid memory storage. An information storage method may include converting input information to a temporal pattern in a form of a spike; and storing the information that is converted to the temporal pattern in a spiking neural network. The storing may comprise storing information by applying, to the spiking neural network, a spike-timing-dependent plasticity (STDP) learning rate that is an unsupervised learning rule.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Korean PatentApplication No. 10-2016-0162934, filed on Dec. 1, 2016, and KoreanPatent Application No. 10-2017-0108470, filed on Aug. 28, 2017, in theKorean Intellectual Property Office, the disclosure of which isincorporated herein by reference.

BACKGROUND

1. Field of the Invention

The following example embodiments relate to a method of selectivelystoring and controlling information using a neural network.

2. Description of the Related Art

Technology for training a spiking neural network based onspike-timing-dependent plasticity (hereinafter, referred to as ‘STDP’)relates to a learning and information processing scheme of the humanbrain. Fundamental research on features and algorithms thereof has beenactively conducted in the field of neurology and electronic engineering.

Dissimilar to an artificial neural network based on a perceptron orfiring rate model, which is generally widely used, a spiking neuralnetwork system uses a spike-form signal as an information transfermedium. Thus, the spiking neural network system may perform highlydifficult functions, such as high-level recognition, inference, etc.,that are considered as actual biological characteristics of the brain.

Methods of storing information in a neural network may not controllearning of new information and stable appendability of existinginformation to be simultaneously performed in a single neural network.That is, such methods have a stability-plasticity dilemma that a loss ofexisting stored information increases to enhance the efficiency ofstoring new information and storing of new information becomesimpossible to enhance the efficiency of maintaining existinginformation.

SUMMARY

At least one example embodiment provides a method and system that mayselect or actively control and change the efficiency of storing newinformation and the efficiency of maintaining existing information ifnecessary.

At least one example embodiment also provides a method and system thatmay outperform a stability-plasticity dilemma by simply changing alearning rate symmetry of a neural network.

According to an aspect of at least one example embodiment, there isprovided a computer-implemented information storage method includingconverting input information to a temporal pattern in a form of a spike;and storing the information that is converted to the temporal pattern ina spiking neural network. The storing includes storing information byapplying, to the spiking neural network, a spike-timing-dependentplasticity (STDP) learning rule that is an unsupervised learning rule.

The storing may include controlling a characteristic of information thatis stored in the spiking neural network by controlling an individualsynaptic stability in the STDP learning rule.

The storing may include controlling a characteristic of information thatis stored in the spiking neural network by transforming a learning ratesymmetry in the STDP learning rule, and the transforming of the learningrate symmetry may include symmetrically or asymmetrically changing asynaptic weight-dependent learning rate in the STDP learning rule.

The storing may include constructing an asymmetric learning rule of ashort-term memory model that forms a volatile memory by controlling anindividual synaptic stability in the STDP learning rule.

The storing may include constructing a symmetric learning rule of along-term memory model that forms a non-volatile memory by controllingan individual synaptic stability in the STDP learning rule.

The storing may include constructing a hybrid learning rule of a hybridmemory model having an intermediate characteristic between an asymmetriclearning rule of a short-term memory model that forms a volatile memoryand a symmetric learning rule of a long-term memory model that forms anon-volatile memory by controlling an individual synaptic stability inthe STDP learning rule.

The constructing of the hybrid learning rule may include constructingthe hybrid learning rule through a linear combination of the asymmetriclearning rule and the symmetric learning rule.

The storing may include storing information through the STDP learningrule that changes a synaptic strength between an input neuron and anoutput neuron in the spiking neural network based on a temporaldifference between an input spike and an output spike.

The STDP learning rule may strengthen the synaptic strength when theinput spike comes before the output spike and may weaken the synapticstrength when the output spike comes before the input spike.

According to an aspect of at least one example embodiment, there isprovided a computer-implemented information storage method includingconverting input information to a temporal pattern in a form of a spike;and storing the information that is converted to the temporal pattern ina spiking neural network. The storing includes storing information byapplying an STDP learning rule to the spiking neural network, and byconstructing an asymmetric learning rule of a short-term memory modelthat forms a volatile memory, a symmetric learning rule of a long-termmemory model that forms a non-volatile memory, or a hybrid learning ruleof a hybrid memory model having an intermediate characteristic betweenthe asymmetric learning rule and the symmetric learning rule bycontrolling an individual synaptic stability in the STDP learning rule.

According to an aspect of at least one example embodiment, there isprovided a computer-implemented information storage system including atleast one processor configured to execute computer-readableinstructions. The at least one processor is configured to convert inputinformation to a temporal pattern in a form of a spike, and to store theinformation in a spiking neural network, and the at least one processoris configured to store information by applying an STDP learning rule tothe spiking neural network, and to control a characteristic ofinformation that is stored in the spiking neural network by controllingan individual synaptic stability in the STDP learning rule.

According to example embodiments, it is possible to further enhance thetechnical flexibility of a neural network system by providing functionscapable of selecting or actively controlling and changing the efficiencyof storing new information and the efficiency of maintaining existinginformation if necessary.

Also, according to example embodiments, it is possible to outperform astability-plasticity dilemma that is one of key issues of a neuralnetwork system by simply changing a learning rate symmetry of a neuralnetwork in a method of storing information in a neural network.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects, features, and advantages of the inventionwill become apparent and more readily appreciated from the followingdescription of embodiments, taken in conjunction with the accompanyingdrawings of which:

FIG. 1 illustrates an example of a structure of a spiking neural networkaccording to an example embodiment;

FIG. 2 illustrates an example of describing a spike-timing-dependentplasticity (STDP) learning rule used in a spiking neural networkaccording to an example embodiment;

FIGS. 3A and 3B are graphs showing a change in an STDP profile forcontrolling learning of new information and sustainability of existingmemory according to an example embodiment;

FIG. 4 illustrates an example of describing a hybrid memory model havingan intermediate characteristic between a short-term memory model and along-term memory model according to an example embodiment;

FIGS. 5A and 5B are graphs showing a result of measuring a temporaldecay of memory according to an example embodiment;

FIGS. 6A and 6B are graphs showing a result of measuring the memoryefficiency of existing memory and new memory in response to appendingnew information according to an example embodiment;

FIGS. 7A and 7B are graphs showing a result of measuring the memoryefficiency of existing memory and new memory in response to sequentiallyappending a plurality of pieces of information according to an exampleembodiment.

FIGS. 8 and 9 are graphs showing memory characteristics for each STDPform according to an example embodiment.

DETAILED DESCRIPTION

Hereinafter, some example embodiments will be described in detail withreference to the accompanying drawings. Regarding the reference numeralsassigned to the elements in the drawings, it should be noted that thesame elements will be designated by the same reference numerals,wherever possible, even though they are shown in different drawings.Also, in the description of embodiments, detailed description ofwell-known related structures or functions will be omitted when it isdeemed that such description will cause ambiguous interpretation of thepresent disclosure.

Hereinafter, example embodiments are described with reference to theaccompanying drawings.

The example embodiments relate to a method of selectively storing andcontrolling information using a neural network, and employ a spikingneural network using a learning and information processing scheme of thehuman brain and a spike-timing-dependent plasticity (STDP) to store andcontrol information in a neural network.

Existing technology for storing information using an STDP learning rulein a spiking neural network does not disclose an active control functionthat enables stored information to have a characteristic of short-termmemory or long-term memory. In detail, existing technologies do notdisclose a method of controlling stored information to have a volatileor non-volatile characteristic or actively controlling previously storedinformation to be overwritten or not to be overwritten by newinformation appended.

The example embodiments may actively control a characteristic ofinformation stored in a spiking neural network by transforming alearning rate symmetry in an STDP learning rule. In detail, the exampleembodiments may readily control the sustainability of stored informationand the effect by new information by symmetrically or asymmetricallychanging a synaptic weight-dependent learning rate of an STDP, which maybe regarded as a simple and new method that overcomes some constraintsfound in the related art by simply transforming a single rule in aneural network model.

That is, the example embodiments may control the stability of an STDPlearning rule to generate a characteristic of information stored in aspiking neural network, that is, a long-term memory that is non-volatileand is not affected by new information or a short-term memory that isvolatile and easily overwritten by new information, or to form a hybridmemory having an intermediate characteristic between the long-termmemory and the short-term memory.

Accordingly, it is possible to construct a system that may activelycontrol and change sustainability and volatility of newly storedinformation by continuously controlling a learning rate symmetry in asingle neural network.

The example embodiments may actively control a characteristic of storedinformation without transforming a physical synaptic circuit even in aneural network. Also, the learning rate symmetry may be continuouslychangeable from the perfect symmetry to the complete asymmetry and thus,may be readily selected and transformed depending on whether it ispossible to construct the hybrid memory indicating an intermediatecharacteristic between the long-term memory and the short-term memoryand a level of stability of information to be stored in the neuralnetwork. Accordingly, it is possible to effectively solve thestability-plasticity dilemma found in the existing neural networks andto construct a memory system further closer to biologically functionalcharacteristics of the human brain.

FIG. 1 illustrates an example of a structure of a feed-forward spikingneural network according to an example embodiment. Referring to FIG. 1,an information storage system according to an example embodiment mayconvert general information 101, for example, a text, an image, etc., toa temporal pattern 102 of a spike representing, for example, a neuralactivity, and may store the temporal pattern in a spiking neural network103.

The information storage system may include a neural network system thatenables pattern learning and information storage by applying an STDP tothe spiking neural network 103. Here, the STDP is a single type ofunsupervised learning rule.

FIG. 2 illustrates an example of an STDP learning rule used in a spikingneural network according to an example embodiment. The STDP refers to alearning rule that changes a synaptic strength between an input neuronand an output neuron based on a temporal difference between an inputspike and an output spike. Referring to FIG. 2, if the input spike comesbefore the output spike (Δt>0), the synaptic strength is strengthened,that is, w increases. If the output spike comes before the input spike(Δt<0), the synaptic strength is weakened, that is, w decreases.

FIGS. 3A and 3B are graphs showing a change in an STDP profile forcontrolling learning of new information and sustainability of existingmemory according to an example embodiment. FIG. 3A illustrates anexample of an asymmetric learning rule of a short-term memory model inwhich an existing memory is easily replaceable with new memory and avolatile memory corresponding to short sustainability of information isformed. FIG. 3B illustrates an example of a symmetric learning rule of along-term memory model in which existing information is maintained eventhough new memory is appended and a non-volatile memory corresponding topermanently maintained information is formed.

That is, as a single example of controlling a stability of a learningrule, a new type of a symmetric plasticity rule different from agenerally used asymmetric STDP may be employed. In this case, there maybe constructed a neural network system that may maintain existinginformation although new information is appended and may form anon-volatile memory having a characteristic similar to a long-termmemory of the human brain of which information is maintainedpermanently. On the contrary, if the symmetry of plasticity in a neuralnetwork is controlled to be in an asymmetric form, there may beconstructed a system that may easily replace an existing memory when newinformation is input and may form a volatile memory of which informationsustainability is relatively short, which is similar to random accessmemory (RAM) of a computer or a short-term memory of the human brain.Also, even in the neural network in which information is stored, acharacteristic of stored information may be actively controlled bychanging a form of the learning rule as above.

FIG. 4 illustrates an example of describing a hybrid memory modelaccording to an example embodiment. Referring to FIG. 4, the hybridmemory model is provided in a form in which an asymmetric learning ruleand a symmetric learning rule are combined at a predeterminedproportion. That is, there may be provided a hybrid memory storagemethod that represents an intermediate characteristic between along-term memory and a short-term memory based on an intermediate formbetween the symmetric learning rule and the asymmetric learning ruleusing principles of a short-term memory model and a long-term memorymodel.

Herein, a change in the STDP learning rate symmetry indicatessymmetrically or asymmetrically changing a synaptic weight-dependentlearning rate while maintaining a STDP kernel to be in an asymmetricshape as shown in FIGS. 3A, 3B, and 4, instead of symmetrically orasymmetrically changing an actual shape of the STDP kernel.

Hereinafter, an example embodiment of a spiking neural network and STDPmodeling is described.

A biologically reasonable spiking neural network may be constructedusing a leaky-integrate-and-fire (LIF) neuron model. The sustainabilityand stability of memory that is formed by apply a symmetric/asymmetriclearning rate STDP based on the neural network may be guaranteed.

The LIF neuron model is represented as the following Equation 1.

$\begin{matrix}{{C\frac{{dV}_{j}(t)}{dt}} = {{{\mathcal{g}}_{L}\left( {E_{L} - {V_{j}(t)}} \right)} + {{{\mathcal{g}}_{j}(t)}\left( {E_{syn} - {V_{j}(t)}} \right)} + I_{noise}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Equation 1, C denotes a membrane capacitance, g_(L) denotes a leakconductance, E_(L) denotes a resting potential, and E_(syn) denotes areversal potential. In addition, g_(j) denotes a voltage gated channelconductance and is determined by the following Equation 2.

$\begin{matrix}{\frac{{d\mathcal{g}}_{j}(t)}{dt} = {{- \frac{{\mathcal{g}}_{j}(t)}{\tau_{syn}}} + {c_{syn}{\sum\limits_{i \in {input}}{w_{ij}{S_{i}(t)}}}}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

In Equation 2, S_(j) denotes a spike fired by a pre-synaptic neuron,w_(ij) denotes a synaptic weight between a pre-synaptic neuron and apost-synaptic neuron, and c_(syn) denotes a size of excitatorypostsynaptic conductance (EPSC) evoked by an input spike. When V_(j)exceeds a specific threshold, for example, −55 mV, an action potentialis generated and a neuron is evoked and, immediately after evocation,V_(j) is reset to the resting potential.

The STDP learning rule for updating a synaptic weight between neurons bya spike timing interval between a pre-synaptic neuron and apost-synaptic neuron is represented as the following Equation 3.

$\begin{matrix}{{\Delta\; w_{ij}} = \begin{matrix}{{{\epsilon_{+}\left( w_{ij} \right)} \cdot k_{+}}e^{- \frac{\Delta\; t}{\tau_{+}}}} & {{{\Delta\; t} > 0},{LTP}} \\{{{\epsilon_{-}\left( w_{ij} \right)} \cdot k_{-}}e^{- \frac{\Delta\; t}{\tau_{-}}}} & {{{\Delta\; t} \leq 0},{LTD}}\end{matrix}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

Here, the STDP learning rate symmetry may vary based on setting ϵ thatdenotes the synaptic weight-dependent learning rate.

An asymmetric learning rate rule of the synaptic weight-dependentlearning rate may be defined as the following Equation 4.ϵ_(asym)+(w _(ij))=w _(max) −w _(ij)ϵ_(asym)−(w _(ij))=w _(ij) −w _(min)  [Equation 4]

A symmetric learning rate rule of the synaptic weight-dependent learningrate may be defined as the following Equation 5.ϵ_(sym)+(w _(ij))=ϵ_(sym)−(w _(ij))=2● min(w _(max) −w _(ij) ,w _(ij) −w_(min))  [Equation 5]

Also, a hybrid learning rule through a linear combination of theasymmetric learning rate rule and the symmetric learning rule rate maybe defined as the following Equation 6.ϵ_(hybrid)(w _(ij))=αϵ_(sym)(w _(ij))+(1−α)ϵ_(asym)(w_(ij)),0<α<1  [Equation 6]

In Equation 6, α denotes a proportion of the symmetric learning raterule.

The neural network disclosed herein may include LIF neurons described inEquation 1 and may include two layers, for example, 50 input neurons and50 output neuron. Here, a synaptic connection probability betweenneurons may be randomly set to 0.2. A synaptic weight may be randomlyinitialized between 0 and 1.

Hereinafter, a learning scheme of a neural network is described.

Initially, all of the input neurons form a temporal pattern with apredetermined length, for example, 100 ms, such that every input neuronfires once with a random timing. Training is performed by repeatedlyfeeding the temporal pattern to the neural network a predeterminednumber of times, for examples 1000 times. To measure the memoryefficiency of the neural network, how consistent an output pattern isacquired may be measured by inputting each of a trained pattern and anuntrained pattern to the neural network. A memory index representing thememory efficiency of the neural network may be defined as the followingEquation 7.

$\begin{matrix}{{MI} = {\frac{1}{N_{pair}}{\sum\limits_{m,{n \in {\lbrack{1\text{:}20}\rbrack}}}\frac{S_{m} \cdot S_{n}}{N_{firing}}}}} & \left\lbrack {{Equation}\mspace{14mu} 7} \right\rbrack\end{matrix}$

In Equation 7, S denotes an output pattern and N_(pair) and N_(firing)denote constants used to normalize the memory index based on a totalnumber of neurons and a number of spikes.

Graphs of FIGS. 5A and 5B show a memory index that varies over time inan environment which noise spikes are introduced with respect to each ofa symmetric learning rate rule and an asymmetric learning rate rule.From this, it can be known that the technology proposed herein maycontrol the memory sustainability. Referring to FIGS. 5A and 5B, aneural network using the asymmetric learning rate rule noticeably decaysby repeatedly introduced noise, whereas the neural network using thesymmetric learning rate rule maintains most of existing memories evenagainst noise.

Graphs of FIGS. 6A and 6B show existing memories that vary in responseto training new memories with respect to each of the symmetric learningrate rule and the asymmetric learning rate rule. From this, it can beknown that the technology proposed herein may control the appendabilityof existing information. Referring to FIG. 6A, in the neural networkusing the asymmetric learning rate rule, existing memories are erased inresponse to appending new memories. Referring to FIG. 6B, in the neuralnetwork using the symmetric learning rate rule, existing memories arenot erased. That is, information of the existing memories and the newmemories may coexist.

Graphs of FIGS. 7A and 7B show the efficiency of existing memories andthe efficiency of new memories that are clearly compared in response tosequentially appending a plurality of pieces of information with respectto each of the symmetric learning rate rule and the asymmetric learningrate rule. Referring to FIG. 7A, in the neural network using theasymmetric learning rate rule, existing memories start to be erased andare completely decayed in response to appending new memories. Referringto FIG. 7B, in the neural network using the symmetric learning raterule, existing memories are accumulated instead of being erased althoughnew memories are trained. Thus, a plurality of input informationpatterns may be simultaneously stored.

Using the same method, it is possible to measure a change in memory ofthe hybrid learning rule represented as Equation 6. FIGS. 8 and 9 aregraphs showing memory characteristics for each STDP form.

The graph of FIG. 8 shows the sustainability of existing memories thatvaries over time in a memory model. Referring to FIG. 8, it can be knownthat a symmetric learning method may sustain stored information during along period and an asymmetric learning method may lose informationrelatively fast over time. In addition, the stored information using ahybrid learning method shows an intermediate characteristic between theasymmetric learning rule and the symmetric learning rule.

The graph of FIG. 9 shows the appendability of existing information inresponse to adding new information. Each of the three learning methodsexhibits a different result. Similar to the sustainability ofinformation, the appendability of existing information according to thehybrid learning method exhibits an intermediate characteristic betweenthe asymmetric learning rule and the symmetric learning rule.

Although the example embodiments describe that a characteristic ofstored information is controlled by transforming the learning ratesymmetry in the STDP learning rule, it is provided as an example only.Any synaptic stability-dependent scheme may be applied. The learningrate symmetry may be transformed unless the synaptic stability isgreatly changed. An accurately symmetric synaptic weight-dependentlearning rate may not be necessarily needed to construct the symmetriclearning rule. If a learning rate is relatively low at a point at whicha synaptic weight is between 0 and 1, it may operate in the same manneras the symmetric learning rule such that a synapse is stable at itsminimum “0” and maximum “1”. Likewise, the same STDP form as in theaforementioned example embodiment may not be necessarily needed toconstruct the asymmetric learning rule. If the synapse is stable at theintermediate synaptic strength, for example, 0.4, it may operate in thesame manner as the asymmetric learning rule.

According to example embodiments, it is possible to further enhance thetechnical flexibility of a neural network system by providing functionscapable of selecting or actively controlling and changing the efficiencyof storing new information and the efficiency of maintaining existinginformation if necessary. Also, according to example embodiments, it ispossible to outperform a stability-plasticity dilemma that is one of keyissues of a neural network system by simply changing a learning ratesymmetry of a neural network in a method of storing information in aneural network.

The example embodiments may be applicable to the overall industry andproducts using a spiking neural network. For example, a balance betweenthe sustainability of existing information and learning of newinformation may be actively controlled by applying the exampleembodiments to an artificial intelligence robot that uses the spikingneural network as a memory. Examples of technology using the spikingneural network may include systems using a neural network as a keyalgorithm, such as deep-learning, cognitive computing, artificialvision, and robot control. In many cases, a learning memory system needsto be mounted. Thus, the methods proposed herein may be significantlyapplied. For example, the learning rule and system disclosed herein maybe applied to design neuromorphic storage device hardware that copiesthe human brain. Through this, it is possible to mimic biologicalcharacteristics of long-term and short-term memories of the brain.

The example embodiments may be applied to many systems using the spikingneural network. The spiking neural network relates to a structure thatstructurally and/or functionally copies the biological human brainregarded as the most excellent artificial system. The spiking neuralsystem may be employed for a variety of fields that use human-levelartificial functions, such as facial recognition, an autonomous drivingvehicle, a smart robot control, and the like.

Also, the example embodiments may be applied to design a storage deviceof a neuromorphic system that is one of technologies currently mostexpected. Once the methods proposed herein are applied, an efficientsystem capable of changing a characteristic of memory may be designed bysimply changing a learning rule without performing a physical operation,such as changing a hardware characteristic or a connection, which maynot be performed in an existing neuromorphic chip or artificial neuralnetwork.

The information storage method according to the example embodiments mayinclude two or more operations based on the description made withreference to FIGS. 1 through 9. The information storage system accordingto the example embodiment may include at least one processor configuredto execute computer-readable instructions. Here, the at least oneprocessor may perform the aforementioned information storage method.

The processing device described herein may be implemented using hardwarecomponents, software components, and/or combination thereof. Forexample, the processing device and the components described herein maybe implemented using one or more general-purpose or special purposecomputers, such as, for example, a processor, a controller and anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a field programmable gate array (FPGA), a programmablelogic unit (PLU), a microprocessor, or any other device capable ofresponding to and executing instructions in a defined manner. Theprocessing device may run an operating system (OS) and one or moresoftware applications that run on the OS. The processing device also mayaccess, store, manipulate, process, and create data in response toexecution of the software. For purpose of simplicity, the description ofa processing device is used as singular; however, one skilled in the artwill be appreciated that a processing device may include plurality ofprocessing elements and plurality of types of processing elements.

For example, a processing device may include plurality of processors ora processor and a controller. In addition, different processingconfigurations are possible, such a parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, to independently orcollectively instruct and/or configure the processing device to operateas desired, thereby transforming the processing device into a specialpurpose processor. Software and/or data may be embodied permanently ortemporarily in any type of machine, component, physical or virtualequipment, computer storage medium or device, or in a propagated signalwave capable of providing instructions or data to or being interpretedby the processing device. The software also may be distributed overnetwork coupled computer systems so that the software is stored andexecuted in a distributed fashion. The software and data may be storedby one or more non-transitory computer readable recording mediums.

The methods according to the above-described example embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations of the above-describedexample embodiments. The media may also include, alone or in combinationwith the program instructions, data files, data structures, and thelike. The program instructions recorded on the media may be thosespecially designed and constructed for the purposes of exampleembodiments, or they may be of the kind well-known and available tothose having skill in the computer software arts. Examples ofnon-transitory computer-readable media include magnetic media such ashard disks, floppy disks, and magnetic tape; optical media such asCD-ROM discs, DVDs, and/or Blue-ray discs; magneto-optical media such asoptical discs; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory (ROM),random access memory (RAM), flash memory (e.g., USB flash drives, memorycards, memory sticks, etc.), and the like. Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter. The above-described devices may beconfigured to act as one or more software modules in order to performthe operations of the above-described example embodiments, or viceversa.

A number of example embodiments have been described above. Nevertheless,it should be understood that various modifications may be made to theseexample embodiments. For example, suitable results may be achieved ifthe described techniques are performed in a different order and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents. Accordingly, other implementations arewithin the scope of the following claims.

What is claimed is:
 1. A computer-implemented information storage methodcomprising: converting input information to a temporal pattern in a formof a spike; and storing the information that is converted to thetemporal pattern in a spiking neural network, wherein the storingcomprises storing information by applying, to the spiking neuralnetwork, a spike-timing-dependent plasticity (STDP) learning rule thatis an unsupervised learning rule, and wherein the storing also comprisesconstructing a hybrid learning rule of a hybrid memory model having anintermediate characteristic between an asymmetric learning rule of ashort-term memory model that forms an emulated volatile memory and asymmetric learning rule of a long-term memory model that forms anemulated non-volatile memory by controlling an individual synapticstability in the STDP learning rule.
 2. The method of claim 1, whereinthe storing comprises controlling a characteristic of information thatis stored in the spiking neural network by controlling the individualsynaptic stability in the STDP learning rule.
 3. The method of claim 1,wherein the storing comprises controlling a characteristic ofinformation that is stored in the spiking neural network by transforminga learning rate symmetry in the STDP learning rule, and the transformingof the learning rate symmetry comprises symmetrically or asymmetricallychanging a synaptic weight-dependent learning rate in the STDP learningrule.
 4. The method of claim 1, wherein the storing comprisesconstructing an asymmetric learning rule of a short-term memory modelthat forms an emulated volatile memory by controlling the individualsynaptic stability in the STDP learning rule.
 5. The method of claim 1,wherein the storing comprises constructing a symmetric learning rule ofa long-term memory model that forms an emulated non-volatile memory bycontrolling the individual synaptic stability in the STDP learning rule.6. The method of claim 1, wherein the constructing of the hybridlearning rule comprises constructing the hybrid learning rule through alinear combination of the asymmetric learning rule and the symmetriclearning rule.
 7. The method of claim 1, wherein the storing comprisesstoring information through the STDP learning rule that changes asynaptic strength between an input neuron and an output neuron in thespiking neural network based on a temporal difference between an inputspike and an output spike.
 8. The method of claim 7, wherein the STDPlearning rule strengthens the synaptic strength when the input spikecomes before the output spike and weakens the synaptic strength when theoutput spike comes before the input spike.
 9. A computer-implementedinformation storage method comprising: converting input information to atemporal pattern in a form of a spike; and storing the information thatis converted to the temporal pattern in a spiking neural network,wherein the storing comprises storing information by applying aspike-timing-dependent plasticity (STDP) learning rule to the spikingneural network, and by constructing an asymmetric learning rule of ashort-term memory model that forms an emulated volatile memory, asymmetric learning rule of a long-term memory model that forms anemulated non-volatile memory, and a hybrid learning rule of a hybridmemory model having an intermediate characteristic between theasymmetric learning rule and the symmetric learning rule by controllingan individual synaptic stability in the STDP learning rule.
 10. Acomputer-implemented information storage system comprising: at least oneprocessor configured to execute computer-readable instructions, whereinthe at least one processor is configured to convert input information toa temporal pattern in a form of a spike, and to store the information ina spiking neural network, the at least one processor is also configuredto store information by applying a spike-timing-dependent plasticity(STDP) learning rule to the spiking neural network, and to control acharacteristic of information that is stored in the spiking neuralnetwork by controlling an individual synaptic stability in the STDPlearning rule, and the at least one processor is further configured tostore the information by constructing a hybrid learning rule of a hybridmemory model having an intermediate characteristic between an asymmetriclearning rule of a short-term memory model that forms an emulatedvolatile memory and a symmetric learning rule of a long-term memorymodel that forms an emulated non-volatile memory by controlling theindividual synaptic stability in the STDP learning rule.
 11. Theinformation storage system of claim 10, wherein the at least oneprocessor is configured to control the characteristic of informationthat is stored in the spiking neural network by transforming a learningrate symmetry in the STDP learning rule, and the at least one processoris configured to symmetrically or asymmetrically change a synapticweight-dependent learning rate in the STDP learning rule.
 12. Theinformation storage system of claim 10, wherein the at least oneprocessor is configured to store the information by constructing anasymmetric learning rule of a short-term memory model that forms anemulated volatile memory by controlling the individual synapticstability in the STDP learning rule.
 13. The information storage systemof claim 10, wherein the at least one processor is configured to storethe information by constructing a symmetric learning rule of a long-termmemory model that forms an emulated non-volatile memory by controllingthe individual synaptic stability in the STDP learning rule.